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Abstract. 

In this note, we give the stochastic maximum principle for optimal control of stochastic PDEs in the general 
case (when the control domain need not be convex and the diffusion coefficient can contain a control variable). 



1 Introduction 

The problem of finding necessary optimality conditions for stochastic optimal control problems (generalizing 
in this way the Pontryagin maximum principle to the stochastic case) has been solved in great generality, in 
the classical finite dimensional case, in the well known paper by S. Peng [4 ]. The author allows the set of 
control actions to be non-convex and the diffusion coefficient to depend on the control; consequently he is led 
to introducing the equations for the second variation process and for its dual. As far as infinite dimensional 
equations are concerned the cases in which the control domain is convex or diffusion does not depend on the 
control have been treated in [TJ [2] ■ On the contrary in the general case (when the control domain need not 
be convex and the diffusion coefficient can contain a control variable) existing results are limited to abstract 
evolution equations under assumptions that are not satisfied by the large majority of concrete stochastic PDEs 
(for instance the case of Nemitsky operators on L p spaces is not covered, see |3]). Here we formulate a 
controlled parabolic stochastic PDE in a semi-abstract way and show how the specific regularity properties 
on the semigroup corresponding to the differential operator can be used to treat the case of Nemitsky-type 
coefficients. The key point is the explicit characterization of the second dual process in term of a suitable 
quadratic functional, see Definition 14.11 
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2 Formulation of the optimal control problem 

Let O C R n be a bounded open set with regular boundary. We consider the following controlled SPDE 
formulated in a partially abstract way in the state space H = L 2 (0) (norm | • |, scalar product (■, •)): 



dX t {x) = AX t (x)dt + b(x,X t (x),u t )dt + ^2a j (x,X t (x),u t )d^ t , t € [0,T],x € O, 

3=1 

X (x) = x (x), 
and the following cost functional: 

J(u)=E [ [ l(x,X t (x),u t )dxdt + E [ h{x,X T (x)) dx. 



(2.1) 



We work in the following setting. 



Hypothesis 2.1 1. A is the realization of a partial differential operator with appropriate boundary condi- 
tions. We assume that A is the infinitesimal generator of a strongly continuous semigroup e tA , t > 0, in 
H. Moreover, for every p e [2,oo) and t € [0,T], e tA (L p (0)) C L p (0) with ||e tj4 /|| LP(0) < C p . T \\f\\ L v{0) 



for some constants C p ^t independent oft and f. Finally the restriction of e , t > 0, to L (O) is 
analytic semigroup with domain of the infinitesimal generator compactly embedded in L 4 (C). 



an 



2. {j3\, . . . ,/?™), t > is a standard m- dimensional Wiener process on a complete probability space (O, £,P) 
and we denote by (J^)t>o its natural (completed) filtration. All stochastic processes will be progressively 
measurable with respect to (J r t)t>o- 

3. b,<r.j{j — 1, ...,m),l : O x R x U — > K and /i:Oxl->l are measurable functions. We assume that 
they are continuous with respect to the third variable ( the control variable ), of class C 2 with respect to the 
second (the state variable), and bounded together with their first and second derivative with respect to the 
second variable. 

4-. the set of admissible control actions is a separable metric space U and an admissible control u is a (pro- 
gressive) process with values in U . 

Under the above conditions, for every control u there exists a unique mild solution, i.e. a continuous process 
in H such that, P-a.s. 



t rt 
o Jo 



X t = e tA x + e ( - t -^ A b(;X s (-),u s )ds+ e (t - s) %(-, X s (-), u s ) d$, t e [0,T]. 



3 Expansions of the solution and of the cost 

We assume that an optimal control u exists and denote by X the corresponding optimal state. We introduce 
the spike variation: we fix an arbitrary interval [t, i + e] C (0, T) and an arbitrary U- valued, J-^-measurable 
random variable v define the following perturbation of u: u\ — v I\i,t+e] (t) + Utl[i,i+e]"(t) an( i denote by X £ the 
solution of the state equation (|2.1I) with control u — u t . 

We introduce two linear equations corresponding to first and second expansion of X e with respect to e (both 
equations are understood in the mild sense). In the following, derivatives with respect to the state variable will 
be denoted b' , b" , a' , a" and 

Sbt(x) = b(x,X t {x),ul) - b(x,X t (x),u t ), Saj t (x) = a 3 (x, X t (x),u e t ) - <7j(x, X t (x), u t ), 

Sb' t (x) = b'(x,X t (x),ul) - b'(x,X t (x),u t ), 5a' jt (x) = <7j(x, X t (x), u e t ) - cr'^x, X t (x),u t ). 



Consider 



dY{(x) = 
Y e (x) = 



AY t e (x) + b'( X) X t {x),u t )-Y t e (x) 



dt + a'jix, X t (x), u t ) ■ Yf(x) d(3 3 t + 6b t (x) dt + 5a jt (x) d(i\ 

(3.1) 
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dZI(x)-- 



AZt(x) +b'{x,X t (x),Ut) ■ Zl{x) 



dt + a'Jx,X t (x),u t ) ■ Zl(x)d(3- 
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b"(x,X t (x),u t ) ■ Yi{xf + 6b' t (x) ■ Yf(x) 




dt- 



\a'!{x, X t (x), u t ) ■ Y*{x) 2 + 6a' jt (x) ■ Y t <(x) 



(3.2) 

We notice that to formulate the second equation in H we need to show that the first admits solutions in L 4 (0). 

The following proposition states existence and uniqueness of the solution to the above equation in all spaces 
L p (0) together with the estimate of their dependence with respect to e. The proof is technical but based on 
standard estimates and we omit it. 



Proposition 3.1 Equations \3.1\) and \3. 2\) admit a unique continuous mild solution. Moreover for all p > 2 



sup (v-e\nY t TL P{ o)) 1/p +^\nmi, { o)) 1/p )< c p> 



sup {m\X\ -X t - Y; ~ Z<\\ 2 H ) 1/2 = o( £ ). 



te[o,T] 

As far as the cost is concerned we set Slt(x) = l(x, X t (x), u^) — l(x, X t (x),ut) and prove that 
Proposition 3.2 



if I Sl t (x)dxdt + A\ + A e 2 + o(e), 
Jo Jo 



where 



Jp 



l'(x,X t (x),u t )(Y t e (x) + Zf(x))dxdt + E / h'(x, X T (x))(Y£(x) + Z^(x)) dx 



■E 



Jo 



o 



l"(x, X t (x), u t ) Y t e (x) dxdt + - E / h"{x, X T (x))Y^(x) 2 dx 



o 



4 The first and second adjoint processes 

The following proposition is special case of a result in [2J: 

Proposition 4.1 Let A* be the L 2 (0) -adjoint operator of A. There exists a unique m + 1-tuple of L 2 {0) 

processes (p, qj), with p continuous and Esup tg [ T ] \pt\ 2 + E J Q \qjt\ 2 dt < 00, that verify (in a mild sense) the 
backward stochastic differential equation: 



-dp t (x) = -qjt(x) df}\ + 
Pt{x) = h'(x,X T (x)). 



A*p t {x) +b'(x,X t (x) 1 u t ) -p t {x) + a'Jx,X t (x),u t ) ■ qjt(x) + l'(x, X t (x), u t ) 



dt 



The following proposition formally follows from Proposition l3 . 2l computing the Ito differentials d J a Yf (x)pt (x) dx 
and d J Z^(x)pt(x) dx, while the formal proof goes through Yosida approximations of A. 



Proposition 4.2 We have 



J(u £ )- J(u) =E 



Jo 



[Sl t (x) + Sb t {x)pt(x) + 5o-j t (x)qj t {x)] dxdt + -A e 3 + o(e), 



whe 



with 



A 3 = E 



Jo 



H t (x)Y t e {xYdxdt + E / h(x) Y^xf dx, 



(4.1) 



(4.2) 



H t (x) = l"(x,X t (x), u t ) + b"(x,X t {x), ut)pt(x) + a'J(x,X t (x), u t )qj t (x), h(x) = h"(x, Xt(x)). 
We notice that the multiplication by Ht(-) is not a bounded operator in H. 
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Definition 4.1 For fixed t G [0,T] and f G L 4 (0), we consider the equation (understood as usual in mild form) 

dY^(x) ^ AY t ^{x)ds + b\x,X s {x),u s )Y t ^{x)ds + a t J {x,X s {x),u s )Y t s ' f {x)dWl s£[t,T], 
Y^{x) =/(»). ^ (43) 

We denote C the space of bounded linear operators L 4 {0) —¥ L 4 (0)* — L 4 / 3 {0) and define a progressive process 
(Pt)te[o,T] with values in C setting for t G [0, T], f,g£ L 4 (0), 



(Ptf, g) = E^' f f H s (x)Y s tJ (x)Y^ a (x)dxds+E Tt f h{x)Y^ } (x)Y*' a (x) dx 
Jt Jo Jo 



a.s. 



(by abuse of language by (•, •) we also denote the duality between L 4 (0) and L 4 ^{0)). 

Exploiting the analyticity of the semigroup generated by A on L 4 (0) we prove the following proposition that 
is the key point for our final argument. 

Proposition 4.3 We have sup te[0 T] E\\P t \\ 2 c < oo. Moreover E|(P t+£ - P t )f, g) \ -> 0, as e -> ; V/, g G L 4 (0). 
Finally for every n G (0, 1/4) there exists a constant C v such that 



\(P t (-Arf,(-Arg)\<C v \\f\\4gUT-t) 



-2 V 



T \ 1/2 

E^lH.^ds) 



(E^l^l 2 ) 1 



a.s. (4.4) 



where D(—A) v is the domain of the fractional power of A in L 4 (0) and by || • || 4 we denote the norm in L 4 (0). 



5 The Maximum Principle 

For u G U and X,p, qi, . . . , q m G L 2 (0) denote 



H{u,X,p,q x , ...,q m ) = 

Jo 



l(x, X(x), u) + b(x, X(x), u)p{x) + Pj(x, X(x), u)qj{x) 



dx 



Theorem 5.1 Let (X t ,Ut) be an optimal pair and let p,qi,...,q m be defined as in Proposition ^. 1\ and P be 
defined as in Definition ^. 1\ Then the following inequality holds P-a.s. for a.e. t G [0, T] and for every v G U: 

H(v,X t ,Pt, qit, ■ • ■ , q m t) - H(ut,X t ,p t ,q u , • ■ ■ , q m t) 

+ \ (P t [<jj(.,X t {-),v) - CTj (-,X t (-),u t )],a J -(-,X t (-),u) - o-j^Xti-lut)) > 0. 



Proof. By the Markov property of the solutions to equation (14.3[) and Proposition 13 . 1 1 we get: 



E / (H,Y a ',Y a e )da + E{hYf,Yj.) = E / (H s Ys° +e ' Yt ° + ' , Ys" +e ' Yt ° + ') ds - 

Jo Jt +e 

o(e). 



nhY^ +e - Yt '« + ',Y^ +e ' Y ^' 



o(e) 



(Pt +eY t e 0+e) Y t e 0+e ) 



We wish to replace Pt +c by Pj in the above that is we claim that E((P to+e — P to ) Y t < g+e , Yf o+e ) = o(e), or 
equivalently that E((P tQ+ , - P fo ) c" 1 /^^, e^Y^) -> 0. 

To prove the above claim we need a compactness argument. A similar argument will allow us to approximate 
P by suitable finite dimensional projections at the end of this proof. 

By the Markov inequality and Proposition [34] if we set K s = {/ G L 4 : f G P>(-A)'', \\f\\ D( - A ^ < CqS^ 1 / 4 }, 
for a suitable constant Co, and denote by fls, e the event {e _1 / 2 (— A)~ v Y t e g+£ G Kg} we get P(f^ e ) < 5. 
Moreover 

E((P in+e - P t0 )e-V%l +e ,e-V 2 Y t l +e ) 

= E[((P t0+e - P t0 ) 6- 1/2 F/ o+e , 6" 1/2 r/ o+e )l os J + E[((P t0+e - P t ) e- 1 /ay4 +ej6 -i/a^ +e)ln4e] 
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By the Holder inequality 

\A\\ < (E||F t0+e -P t0 |||) 1 / 2 (E|| e - 1 /2^ +£ ||8)i/4 p( ^ £) i/4 j 

and from the estimates in Proposition 14.31 we conclude that \A\\ < cP(f2^ e ) 1 / 4 = O^ 1 / 4 ) 
On the other hand, recalling the definition of 5 , £ , \A%\ < Esup /6Xa |((Pt 0+£ - Pt Q ) {-A)^f, (-A) 7 ) f)ln Sc |. 
Since K$ is compact in L , it can be covered by a finite number Ng of open balls with radius 6 and centers 
denoted //, i = 1, . . . , iVj. Since £)(— A) 17 is dense in L 4 , we can assume that // G £>(— A) 1 '. Given / e J^, let 
i be such that ||/ — ff\\i < then writing 

<(P t0+e - P t0 )(- J 4)"/, M)"/) = ((^o+e - P to )(-A)V/, i-A)Vt) 

-((P t0+e - P t0 )(-A)"(/ - //), (-Ay(f - //)) + 2((P t0+e - P t0 )(-^)V, - /?)) 

and taking expectation, it follows from (|4.4p that 

< $>|<(P to+e - P f0 )(-A)V/, (-A)"//>| + <T -t Q - e)-^[8 2 + <5 3 / 4 ], 
»=i 

and by the second statement in Proposition 14.31 we conclude that 

limsup \A\\<c{T- tvT 2ll [5 2 + <5 3/4 ]. 

Letting S -> we obtain \A\\ + -> and the proof that E((P to+e - P to ) e-^ 2 Y t e 0+e , e _1/2 ^ +e ) -> is 
finished. 

We come now to show that 

E<p fo r/ o+e ,y/ o+e ) =e / f ° +e (p^v J ( s ,.),<5V,( s ,-))^ + o(6). 

If we treat A and Pt as bounded operators in H we get, by ltd rule: 

E(P t0 F/ 0+£! Y& +e ) = 2E [° (P to Y*, (A + b'(X t ,u t ))Y s ')ds + E [ ° (P^a^s, •), <5V,( S , •)) d S 
•/to -/to 

and the claim follows recalling that E| Y t e | 2 = O(e) and the "continuity" of P stated in Proposition ^. 31 
The general case is more technical and requires a double approximation: A by its Yosida Approximations 
and P by finite dimensional projections P^(uj)f := ^Zi^i^t^)^) e .7')( e i> /)2 e i> f € £ 4 , where (ej)j>i is an 
orthonormal basis in L 2 which is also a Schauder basis of L 4 . 

The conclusion of the proof of the maximum principle is now standard (see, e.g. [2J, [5] or [3]). We just have to 
write J(u e ) - J(u) using (|iT]) . flU) and ([BTT]) , to recall that < e" 1 (J(u e ) - J(u)) and to let e -> 0. 
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